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ABSTRACT 

A procedure for estimating reliability in a factor 
analytic context, vben reliability of the extracted factors is not £n 
emphasis, is identified. The procedure is an extension of Dressel's 
vcrk and might be applied in attitude measurement. It assesses bov 
homogenecus the weighted original item responses are, uhen they are 
scored fcr pattern congruity. Four data sets generated to simulate 
different sample sizes, numbers of variables, numbers of variables 
per factor, and proportions of randomly generated error, yere 
analyzed with alpha factor analysis followed by rotation to the 
varinax criterion. Subsequently, both types of ccnslstancy estimates 
were calculated. Besults indicate that the tvo indices are sensitive 
to different data characteristics. The alpha coefficients are 
primarily sensitive tc the variance each factor explains, as 
reflected by the number and magnitude of the salient pattern 
coefficients associated vith each factor. The Dressel ccef f icient s, 
hovever, are sensitive to the proportion of error embedded in the 
original data, and are less sensitive to the number of salient items 
associated uith each factor. (BL) 
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Educational researchers have long recognised the 
usefullness of factor analytic procedures ♦ Before 
widespread access to computers was achieved, factor analysis 
was infrequently performed , because factor analytic 
techniques require rather forbidding calculations ♦ The 
promulgation of computers brought a corresponding increase 
in the use of factor analysis, however* In fact, for a 
brief period researchers almost seemed to factor routinely 
virtually any data which they collected* This abuse abated 
somewhat with an increasing recognition that knowledge of 
the dimensions underlying variables is not per se valuable, 
and several didact ic demonstrations that identified factors 
are not necessarily '*true" primary dimensions* For example. 
Overall (1 964) factor analysed book measurements and 
isolated the dimensions of size, obesity, and squareness, 
rather than the more commonly accepted primary dimensions of 
heighth, width, and thickness* 

Today factor analytic methods are more sensibly applied 
only in the context of specific theory regarding the 
dimensionality of measures ( cf * Gruilford , 1 967 ) , as a 
valuable tool for assessing validity (Nunnally , 1 967 , pp - 
100-101 , Thompson, in press) , and in some experimental 
research as a prelude to various ANOVA procedures and their 
analogues (Morrow and Prankiewics, 1979) • Factor analysis 
can te practically employed to examine the dimensions 
underlying several of the "factorable entities" discussed by 
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Cattell { 1 966 ) . Thus researchers have employed factor 
analys is to factor variables (Thompson and Rucker , in 
press), to factor people (Korb and Prankiewicz, 1976), and 
to factor occasions of measurement (Jones , Miller , and 
Thompson, in press). 

However^ as Cronbach (1951, P* 297) haa emphasized, 
"no factor analysis can be interpreted without some 
appropriate estimate of the magnitude of the error of 
measurement." Researchers typically rely on alpha factor 
analysis (Kaiser and Caf f rey, 1 965 ) to generate these 
estimates . Alpha factor analysis assumes that variables 
rather than persons have have been sampled, and that the 
researcher is interested in estimating how closely obtained 
results correspond to the results that would have been 
obtained if the entire population of variables measuring a 
construct had been used in place of only a sample of the 
variables from this universe. Of course, researchers rarely 
employ every available measure from any universe of 
measures, because the number of variables which measure most 
constructs is usually very, very large indeed. Thus, this 
correspondence estimate is frequently important, because 

there is no practical testing problem where the 
items in. the test and only the items constitute the 
trait under examination . We may be unable to 
compose more items because of out limited skill as 
testmakers but any group of items in a test of 
intelligence or knowledge or emotionality is 
regarded as a sample of items. (Cronbach, l95lt 
p. 507) 
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This paper , however , rests upon a different view of 
educational measurement. Researchers do not always wish to 
generalize from a sample of measures to the entire universe 
of possible measures . This occurs , for example , when a 
given set of measures has been frequently used in past 
research, and the researcher is primarily interested in 
generalizing across studies rather than to an abstract 
population of all possible measures. Clearly, alpha factor 
analysis is helpful when measurement and theoretical issues 
are addressed, but the reliability coefficients which it 
generates are couched in a perspective which may not always 
be appropriate . 

The inquiry reported here was conducted in order to 
identify a procedure for estimating reliability in a factor 
analytic context when reliability of the extracted factors 
is not an emphasis. The inquiry was also conducted for the 
further purpose of estimating how adequately the procedure 
performs. The procedure is an extension of Dressel's (1940) 
work, smd might be applied in attitude measurement* 

Psychometrics of the Methods 

"Alpha" is the name which Cronbach (1951 ) gave to a 
coefficient whiah is sensitive to the internal consistency 
of data generated by a set of variables. The primary 
advsmtage of alpha over split-half consistency estimates is 
that alpha is unique for any data set while sp-lit^^half 
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consistency estimates vary over different arbitrary splits 
of the data. One interpretation of alpha is that it 
represents the mean of all possible split-half coefficients 
and is the general case for which the Kuder-Richardson 
estimate is a special case . More complete discussion of 
alpha is available elsewhere (Cronbach, 1951 , McKennell, 
1977, Tyron, 1957). 

In the context of factor analysis, Kaiser and Caffrey 
(1965j P- 6) suggest that alpha is expressed by (1), as a 
function of the v number of variables in a measure and the 
eigenvalue (x) for the j[-th factor. Although there is some 
doubt (McDonald, 1970, pp. 16-20) that this value can always 
"be interpreted as a generalizability coefficient (Cronbach, 
Grleser , Nanda, and Rajaratnam, 1 972) , the value is certainly 
an estimate of factor reliability. Grlass (1966) has argued 
that the results obtained with alpha factor analysis are 
valuable in any research utilizing fallible measures 
(always!?) . Conceptually, alpha is a weighted average of 
the intercorrelations of the factored measure's items. 

Insert Eq^uatlon 2. about here 

However, an alternative estimate of consistency may be 
examined in a factor analytic context . This approach is 
based on an extension of (2) (Dressel , 1940, p. 509 ) , in 
which aj^ is some weight applied to the scores of each n 
persons on each ^ items, and 2^^^^ proportion of the n 
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persons who "succeeded" on the item. This formula is 
helpful to researchers who wish to estimate reliability for 
cognitive measures , which have right-wrong answers , when 
different questions "count" different amounts. 

Insert Equation 2 about here . 

In a factor analytic context, a^ might be defined as 
the magnitude of a factor pattern coefficient of item i_ on 
the 2rth factor. How then should "success" be defined if 
the measure is attitudinal? Success might be defined by 
examing the data provided by individual subjects. An 
intra-subject mean (or median) can be calculated for each 
person by pooling responses across items . The item i_ 
response of the ^-th individual might be considered 
successful if the item varies about the n-th subject *s mean 
in a manner which reflects the pattern of the salient 
pattern coefficients for the j[-th factor. Test statistics 
can be applied to determine pattern coefficient salience , 
but the identified magnitudes would largely be an artifact 
of sample size. A more reasonable strategy might involve a 
"meaningfullness" criterion. This value would vary from one 
study to the next, but generally an item might be considered 
salient to a factor if the item had a structure coefficient 
greater in magnitude than .5. A structure coefficient of .5 
indicates that the item shared at least nine percent of its 
variance with the factor. 
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Conceptually the proposed extension of Dress el ' s 
formula assesses how homogeneous are the weighted original 
item responses, when they are scored for pattern congruity. 
It is important to emphasiise that the index assesses the 
weighted consistency of the original responses, and not the 
reliabilities of factors. 

Method & Results 

The inquiry involved generation of Monte Carlo-type 
data for situations involving the Table 1 parameter 
combinations* Pour data sets were generated to simulate 
different sample sises , numbers of variables , numbers of 
variables per factor, and proportions of randomly generated 
error. All four data sets were analysed with alpha factor 
analysis followed by rotation to the varimax criterion. 
Subsequently, both types of consistency estimates were 
calculated; these are presented in Table 2. 

Insert Tables 1^ & 2 about here . 
Discussion 

Table 2 suggests that, as expected, the two indices are 
sensitive to different data characteristics . The alpha 
coefficients are primarily sensitive to the variance each 
factor explains, as reflected by the number and magnitude of 
the salient pattern coefficients associated with each 
factor. This could be prob?.ematic , If researchers did not 
carefully scrutinise the original data for reliability. 

s 
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This study and several others (cf. Horn and Knapp, 1975) 
suggest that high alpha coefficients can be produced even by- 
unreliable or random data. The Dressel coefficients , 
however, are sensitive to the proportion of error embedded 
in the original data, and are less sensitive to the number 
of salient items associated with each factor- Consequently, 
these results suggest that the Dressel type procedure has 
some promise as an estimate of the weighted pattern 
congruity of the original data. 

Of course, much refinement and further examination must 
be completed before the procedure can be considered usable. 
However, if the procedure ultimately proves to be. viable, 
several advantages might be realised* First, the technique 
could be employed to identify subjects whose responses 
deviate dramatically from model expectations. Some research 
(McCollum and Thompson, 1980) suggests that response style 
and bias errors in attitudinal measures may be much more 
serious than is generally recognised, and one strategy for 
minimising and detecting these influences when ipsative 
measures can be employed has been identified . But the 
Dressel type procedure might usefully be employed when 
measurement is normative. The procedure might be utilised 
to identify "misfitting" persons in a manner analogous to 
latent trait theory identification of "misfitting*' persons 
(Wright and Stone, 1979)- 
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Secondly, the technique offers some hope that the 
consistency of data can be examined in a factor analytic 
context when the researcher does not wish to generalise to a 
universe of all possible construct measures. This will be 
helpful when methods other than alpha factor analysis are 
utilized, or when the factored matrix of associations is a 
variance-covariance rather than a correlation matrix 
( Thompson and Staple ton , in press) - Ultimately, the 
procedure may provide a helpful adjunct to the use of factor 
reliability (Kaiser and Caff rey , 1 965 ) and invar iance 
(Gorsuch, 1974. p* 309) coefficients* 
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TABLE I 
Parameter Combinations 

^ Error 
N V V/Factor Proportion 

400 15 5,4,3,2,1 moderate 

100 20 5,4,3,2,1 moderate 

100 20 5,4,3,2,1 large 

60 25 5,4,3,2,1 moderate 

^or first 15 items before error introduced. 
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TABLB II 
Homogeneity Coefficients 



N of 

Situation Factor Salients Alpha Dressel 



I 


5 


.86 


.70 


II 


3 


.78 


.71 


III 


2 


.72 


.47 


IV 


4 


.54 


.59 


V 


0 


.12 




I 


5 


.84 


.67 


II 


3 


.74 


.70 


III 


4 


.71 


.52 


IV 


2 


.62 


.46 


V 


3 


.58 


.08 


VI 


1 


.35 


.01 


VII 


1 


.28 


.01 


VIII 


2 


.11 


.10 


I 


6 


.82 


.32 


II 


3 


.73 


.63 


III 


2 


.69 


-.01 


IV 


1 


.63 


.01 


V 


2 


.61 


.37 


VI 


1 


.51 


.01 


VII 


2 


.30 


.11 


VIII 


2 


.14 


-.11 


I 


4 


.84 


.65 


II 


6 


.78 


.62 


III 


3 


.74 


.46 


IV 


4 


.68 


.34 


V 


5 


.62 


.33 


VI 


4 


.59 


.16 


VII 


3 


.50 


.30 


VIII 


1 


.34 


.02 


IX 


2 


.21 


.12 
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